SUMMARY_21

By Kate Bulkley

Picture This_21 celebrated its fifth edition back in person at the National Film School of Denmark as well as with a large online audience from Aarhus, Viborg, Odense, and Copenhagen.

The growth in virtual production (VP) stages across Europe is fast increasing and the new production methods are being used more extensively in both film and TV production. The increase in usage of VP techniques is influencing both the creative and the workflow decisions being made and changing the business. Therefore, the Nordisk Film Fonden initiated and funds the conference as part of its support for embracing digital technologies to help grow the Danish production sector and to keep it relevant.

Session 1:
Virtual Production – What Producers Need to Know About It
Niko Remus, a post-production supervisor and the Head of Studies at A Post Lab, underlined in the opening session how VP is flipping the traditional workflows of film making on their head. In the VP production schedule visual effects (VFX) moves from the end of the process in post-production to the very front of the production schedule. This seismic shift in production schedule means that Art Departments and VFX artists have to understand and work with Pre-visualisation techniques and collaborate with Virtual Art Department teams, starting at the beginning of production.

Watch the session:

This shift has accelerated especially in the last ten years as gaming technologies have influenced visual effects techniques. He reminded the audience that the first time that a computer game was used on screen was in the film ‘Soylent Green’, which was released in 1973. Remus charted the growth and changes in green screen technology and how game engine technologies have influenced both the pre-visualization work in film as well as VFX in general.

As a post-production professional, he showed examples of how LED walls and background plates greatly expand the flexibility of how scenes are filmed, especially when there are multiple takes required.

“A studio set with this kind of VFX plate technology means that reflections on windows and bottles for example would be really hard to create in VFX. These are now done as in-camera effects, so your VFX is done essentially at that moment,” he stated.

Remus underlined that the tasks that are now needed in a production go beyond the traditional VFX crew to embrace new skills and specifically VP skills. VFX is typically 10-15% of the total budget, but studio costs in VP are typically more expensive because of:

1. LED panel costs

2. shooting asset plates

3. using playback engines

4. a differently skilled crew.

All this is scalable, explained Remus, but he admonished producers and executive producers that there is need for different cash flows to fund VFX much earlier in the production process.

“The industry is set to have VFX and production overlap so we see what we can shoot and then we modify as we shoot and then we reshoot,” said Remus.

He ended the session saying that there is green aspect to using more VP techniques especially by cutting out travel.

Session 2:
Dark Bay – A Revolution for Studio Shoots
Philipp Klausing is film producer and Managing Director of Dark Ways, the German independent production company behind the Netflix supernatural hit series ‘Dark’. He joined the conference virtually to talk about how the company created a purpose-built virtual studio in Babelsberg Studios for its next Netflix project, ‘1899’, a drama about immigrants on a boat journey across the Atlantic from Europe to the USA.

Watch the session:

The Dark Bay studio is Europe’s largest state-of-the-art virtual studio space with one of the world’s largest LED installations and a revolving stage that is 21m in diameter and can carry up to 25 tons of weight. Despite the circa $10 million cost of building the new volume stage, which was part-financed by Netflix, it allowed the production of ‘1899’ to be made on a much more cost-efficient budget and to shoot throughout Covid. The stage obviously cost more than a single production could support, but owning the stage helped make ‘1899’ at a more reasonable cost.

The TV series used a “full VP schedule” and although VP can mean many things, Philip focused on how they built the Dark Bay studio to use mixed reality stages (or volumes) to augment reality with game engines running the LED screens and actors working in a dressed ‘real environment’ foreground.

Dark Ways spent most of 2020 on “endless Zoom calls” and after realising that they could not use the Manhattan Beach VP studio in LA – where ‘The Mandalorian’ was shot – because of Covid travel restrictions, they decided to fund and build a new stage through partnerships with camera vendor Arri, the Dutch company Faber AV which supplied displays and Framestore of London for interactive lighting and LED panels. Studio Babelsberg and Netflix also participated in funding the project, the latter through a minimum guarantee. Construction of the stage began in January 2021 with an eight-week build. The stage was completed in early March and calibration and lighting took another few weeks with production beginning at the end of March 2021.

There are 55-metre-wide and 7-metres high LED display walls and a partial LED ceiling to have better light control, but you need to pre-grade it, which is another cost upfront in the production. “The brighter the light you need the harder it gets to do in the volume,” said Klausing.

There is a rain rig and a revolving ‘turntable’ stage that can carry 30 tons of sets. There is a “brain barn” with eight work stations that feed into a server room with 20 render nodes that can route images back to the LED walls. Having enough graphic power is key so that your camera moves are not impacted. Resolution on the LED displays needs to be at least 2.3 pitch pixel and being able to adjust colours is also “super important” because colourimetry is crucial to capturing as much as possible in camera, explained Klausing. For ‘1889’ they captured 70% of their shots in camera, which is a ‘top of the industry’ standard.

Klausing says that VP needs to shift formerly post-production work to the beginning, which means “prep commitment” including embracing real-time game engines early in the process to build the content that you will use later. This meant starting to work eight months before they started shooting. He pointed out that in VP you are building not just a real set but a whole world around it, too.

Coordinating the visual effects department and the art department early is key and his learning is that the production designer needs to deliver the entire set, both the virtual and the real sets. Two-three months per set is ambitious but possible and then the stunt requirements and technical requirements can be worked out with VR goggles. “Fixing it in prep is the key,” he said. If you do all this then the actual shoot becomes ‘easy’ because the majority of the work has been done. The idea is to catch as much as you can in camera so most of the shoot can be handed over to the editor without a lot of expensive post production work. For ‘1899’, they had 100 total shooting days with 66 in the volume stage across 17 sets.

The game changer for this VP methodology will be the creation of libraries that can be pulled from. This would make VP a lot easier and cheaper! Klausing feels that “VP is a methodology not a rental product” and that local language markets will benefit most from volume stages because volume technology gives more options for a better production.

SESSION 3:
Virtual Pre-Production and The Role of The Virtual Art Department
In session 3, Felix Jorge, CEO, creative director and co-founder of Virtual Art Studio Happy Mushroom, talked about the process and the products that make up a virtual art department. He discussed tools including his own ‘Mushroom Playground’ and tools that allow virtual scouts, virtual blocking and 3D conceptualizations. He says that tech visualization teams should in the best of all worlds attach themselves to a production’s art department.

His main message revolved around the importance of teamwork between virtual and traditional production and how the workflows have changed and continues to evolve. He also showed examples of how quickly virtual environments can be created, including lighting, which saves time and money in post-production.

SESSION 4:
Collaboration in Creation
Session 4 was a deep dive into ‘The Search’, a transmedia project with Director Michael Madsen, Feileacan McCormick, former architect and artist, and Jonas Frederiksen, a producer at Nimbus Film. They shared their vision for how to talk about climate change through a story about populating and designing an intergenerational spaceship. The project began with talking to a psychologist and with members of the public about the idea of joining such a spaceship for real. They created an installation at the Louisiana Museum outside Copenhagen with a personality test for members of the public to ‘apply’ to go on such a spaceship.

As a documentary filmmaker, Madsen is trying to bridge the gap between what we can see now and how to conceptualise such a film about something that must be imagined. McCormick talked about how to make the ‘sets’ for the film be more symbolic and more theatrical in feel than a photorealistic production. They are using game engines and 3D modelling and an iPad camera but they are not trying to create a full 3D, virtual environment. Producer Frederiksen explained that this film is “out of his comfort” zone and how to organise the feature film from a workflow perspective.

Madsen is hoping that the film will explore how the people who are born on the ship will act and feel given that they will have no memory of living on Earth.

The film should begin shooting in 2022.

SESSION 5:
The Virtual Production Behind “Stowaway”
Session 5 brought Postproduction Supervisor Niko Remus together with Jannicke Mikkelsen, film director and cinematographer to talk about the VP involved with Netflix film ‘Stowaway’ (2021) directed by Joe Penna. The film depicts a perilous journey to Mars which has been compromised by a stowaway on board the ship.

Mikkelsen created a graphic to show all the virtual elements for the pre-visualisation alongside the ‘real’ grips and other ‘in real life’ components. “My biggest challenge is virtual physics,” says Mikkelsen who was the virtual cinematographer on the film.

“You need to work inside the game engine and even though it is not a real camera it comes close and to be honest we all got pretty sick the first week before we locked down the ship rather than having it rotate,” said Mikkelsen.

Cost also must be considered.

“Reality is relative when you shoot in virtual production. We can do anything, but we may not have the budget for it. My job is to hand over to the DP so they have a manageable set. You can go Michael Bay but you may not have the budget for it,” she continued.

“Stowaway has the pre-visualisation and the post moving closer together but not really flipping yet,” observed Remus. “All the shots done in the virtual pre-vis had to be created later in one of the two studios where the film was shot. In the pre-vis it was important to understand the limitations of the set and how high we could go and what angles we could shoot. This was important to find out before we got into the studios.”

After the pre-vis, the sound designer created some soundscapes to give a feel, which was used in pre-shoot editorial to fine-tune the shots.

“With the pre-vis we were finding the rhythm and the voice of the movie. If it is fast editing or slow editing depends on the actors and the music so that is why sound design is so important to bring in from the very start,” said Mikkelsen.

During the session they talked at depth about how pre-visualisation can allow a lot of experimentation and it is immediate as opposed to waiting when it will cost more. Even so, there was still a lot of VFX in Stowaway after the pre-vis. Remus explained that after the pre-vis, the scenes was shot in a studio, but some elements – the sun, for example – could not be shot and was created in VFX.

Mikkelsen said that the DP on the film had to be convinced to use the virtual tools but once he used it, he loved it.

“Letting the director and the director of photography create a movement without setting one single key frame is very important. The movement was like operating a real camera,” said Remus as a closing remark. “Here is the space, let’s move around in it and find the angles.”

SESSION 6:
My Journey Into Real-Time Filmmaking With Virtual Production
Session 6 was led by Hasraf ‘HaZ’ Dulull, director and producer, who has worked in both visual effects and in video games. In 2018 he had a breakout feature film called ‘The Beyond’ and has been working on Disney’s action-comedy series ‘Fast Layne’.

Dulull gave an energetic presentation about his adoption of game engines and his next film ‘RIFT’, a sci-fi anime rendered entirely in Unreal Engine that will be released in 2022.

Watch the session:

Dulull got into game engines in 2019 when he first downloaded the Unreal Engine and started playing with it and experimenting.

“I don’t think of it as pre-vis but as pitch-vis, because it looks so good that I can create a version of a trailer or a movie to convince financiers to back my film! It reminds me of Dogma, the movement founded in Denmark, that gives the power back to the director/creator and away from the studio system,” said Dulull.

He showed a short clip of a proof-of-concept animation called ‘Battlesuit’ that he made in April 2020, which costed about £12,000 for a comic book adaption. This got the attention of Epic Games and Nvidia, the chip maker and made it possible for him to be taken seriously as an animation filmmaker.

“If you want to have a real career in filmmaking as a creator you need to have a business mindset and an understanding of the market. During Covid live action shut down and demand for animation rose exponentially,” said Dulull.

He also told the audience to “promote the hell out of your work. Don’t wait until it is perfect.”

Like several speakers, Dulull says the production roles in films are changing because of the potential of game engine tools, for instance how his role as director changes when you are working with game assets. “Get out of your comfort zone and live on the edge,” said Dullul. For example, he was able to capture reference video while the voice actor was reading the script, in order to get the facial data that he used for the animation. The point is that there are so many tools available now for free or near-free on the internet that allow great creativity at much lower cost. The tech actually enables more creativity.

SESSION 7:
LED Virtual Production Learnings from Danish Drama Series
Session 7 was a deep dive into the practical issues of LED virtual production on the Danish HBO Max original drama series ‘Kamikaze’ and Lars von Trier’s ‘Kingdom Exodus’ produced by Zentropa Entertainment.

VFX & Post Producer Mikael Windelin led the session discussing what is possible in VFX within the constraints of the Danish drama budget. VFX Supervisor Peter Hjorth talked about his initial reluctance to use full LED rear projection technology partly because of fears about potential limits to long lenses and because there was a lot of other issues with the production that he had to deal with it. His learning was that it is important to ask questions and reach out to others for help. For example, he was told: “If you have less than 10 shots, do it on a green screen, don’t try to do it on an LED.” He also recommended that there should be someone dedicated to pre-production of any LED components and that more prep time is needed to get this right. There is also a lot of attention that needs to be made to the colour of the lighting.

“There were some challenges about the colour balance between different sources of light. It’s basically like doing CGI. You have to be super serious about your colour pipeline because these kelvin degrees are not like a linear thing that you can move around the scale,” said Hjorth.

Post & VFX Producer Jacob Clemmensen talked about the fact that Covid restrictions meant that for ‘Kamikaze’ they couldn’t travel to Seoul. Instead they created a set piece and did the driving scenes using plates bought on the internet from a driving plate company, a splinter unit that went out and shot the city of Seoul and the main actor entering the set piece in the studio. “We cheated a lot and it worked pretty well,” said Clemmensen. He admitted that he has not used Unreal Engine for driving scenes, yet, but he would like to.

The session had the professionals talking through how to ‘get the shot’ through a combination of different technologies and good communication on set and the use of good sense and experience. Clemmensen also underlined the importance of planning, testing and getting the proper references for the shot with an iPhone.

“We didn’t use a real pre-vis, but we did an animatic that showed the pacing of the scene and how long the cuts could be, the framing and how all this would work together. Doing this is super helpful,” said Hjorth.

“There is a willingness to put money and time into pre-vis now. But the traditional way of funding Danish productions makes it difficult spending money earlier,” said Windelin.

Hjorth countered that “pre-vis is great if it is not coming out of the VFX budget!”

A huge thank you to the speakers and participants, who joined us for PICTURE THIS_21. Our gratitude also goes to our collaborators and sponsors from Film Workshop Copenhagen, The National Filmschool of Denmark, Danish Producer's Association, The University of Aalborg – CPH, The Association of Danish Film Directors, Danske Scenographers, Danish Writers’ Association, Producentrettigheder Danmark, Creative Europe Desk Denmark and Creative Europe Desk Berlin-Brandenburg.

PICTURE THIS_21 was initiated and funded by NORDISK FILM FONDEN and organized by Vision Denmark.

All rights reserved to the original copyright holders.